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Extract text boxes; apply text 
recognition; Generate role- 
name vs actor-name list 
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Extract face Wes; apply face 
recognition (eigenfaces, model 
based, etc.) Generate face 
models. 



If closed captioning not 
available perform audio 
segmentation and speech to text 
conversion 
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name Ni in the list role -name- 
actor-name list do the following 




Search for role.-name in the 
transcript ; . A - 



Using-'the time code, search the 
corresponding video frames in 
the same or following shots for 
face models 
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Assign equal certainty that the 
name is associated with one of 
the found faces. 
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Eliminate faces which 
pronounce the name (use lip 
syncing method) 



Scan through the rest of the 
transcript and update certainties 
for each next appearance of 
role-name 
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If the certainty for a face model 
and role-name association is 
above a threshold 




yes 



Go back and backpropage by 
marking all the segments where 
this face appeared 




Store role -name and- actor name 
and face models in a database 
for future retrieval ~ 



If Ni is the last element of the 
role-name list 
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Son the list of face models and 
keep the n most promising face 
models for future use in the 
database 
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Role questions: who acts as in the 
role of Mr/Ms. X? When did Mr 
X appear? Summarize the movie 
rrom MrX's point of view 
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Actor questions: who is Mr/Ms. 
Y v in the movie (which role)? 
What did he say? Summarize the 
movie from Mr Y's point of view 



Database: For each movie: Role- 
name with actor name list, For 
each role name: video segments 
(from to)that it appears in, for 
each segment, the sentences 
spoken by the role 



Database of directors/movies/actors 



no 



Output: name, or sentence 
ox video segment, or a list 
of video segments 
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